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To assess public response to five science programs shown on National 
Educational Television, questionnaires were sent to selected high school and university 
science teachers, and two programs were shown in private screenings to sample 
groups of adults and students, responses to the questionnaires showed that a large 
majority of those answering had a favorable impression of the scientific presentation 
of the programs, but focussed concern on the ability of the lay public to understand 
the scientific material involved. In the private screenings, because there were no 
experimental tests to determine learning systematically, all learning over chance was 
taken to be significant, and on this criterion the films were felt to produce significant 
information gain by the viewers. It was felt by all groups that the science programs 
encouraged and maintained interest in science. (BB) 
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SCIENCE PROGRAMMING AND THE AUDIENCES FOR PUBLIC TELEVISION 



An Evaluation 
Of Five Programs in the 



RET "SPECTRUM" 



Series 



Questionnaires returned by science teachers across the country and reaction 
forms completed by selected high school, college, and adult learners, were 
analyzed to judge the impact on and acceptance by public television audiences, 
of five half-hour programs broadcast in the "Spectrum" science series, produced 
with financial support from the National Science Foundation. A general sig- 
nificant knowledge gain was achieved by high school, college, and adult 
learners; learners and science teachers surveyed reported a satisfaction that 
science programs should be broadcast for the general public and that in addition 
to gaining a high rate of acceptance by laymen, the programs encouraged and 
maintained interest in science. 
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INTRODUCTION 



This report is an evaluation of the impact and acceptance of five 
programs produced with financial support from the National Science 
Foundation and broadcast in April and May, 1969, b • National Educational 
Television* as part of the continuing weekly science series, "Spectrum. 11 



The Programs 

1. Exploring the Universe— In Radio and Light (April 30)** 

New developments in radio astronomy and optical astronomy to 
further understanding of the nature of stars, galaxies, quasars, 
and past and present of the universe itself. 

2. Changing the Weather (May 7) 

Recent research in meteorology, focusing on the problems and 
methodology of altering storms and controling climate. 

3. The Trembling Earth (May Ik) 

Current issues in seismology to discover the structure of the earth 
and major geological processes in predicting earthquakes. 

4. Stop or Go— An Experiment in Genetics (May 2l) 

Experiments in understanding the factors which govern heredity, 
chemical language, and pursuing the ability to alter genetic 
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formulas. 



5 * A Visit with Harold Urey (May 28) 

Hie Nobel-Prize-Winner, chemist, physicist, creative thinker, and 
teacher discusses the major influences on his life and the con- 
clusions he has reached about science and our world. 



Originally, these programs were not broadcast in sequence. Two of 
them were aired late in 1967, two early in 1968, and one in the spring 
of 1969. In order to make possible the evaluation reported here, the 
MET program department arranged to have the first four programs re- 
peated at weekly intervals, following the premiere of the latest one. 



**lhe dates ^(Wednesdays) listed are those of the network "feed" at 
o :00 p.m., E.D.T., but, as is customary, individual stations retained 
the option to air the programs at those times or to tape them for 
broadcast at a later time. Many stations also scheduled a repeat 
broadcast within a few days. We can assume with confidence that 
virtually all the MET affiliates did air the five programs at some time 
during the week of the network feed. 
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Method of Investigation 



As outlined in the original NET proposal to the National Science 
Foundation, the evaluation was planned in two major phases: (l) a 

mail questionnaire (see Appendix A), addressed to teachers of high school 
and college science, elicited responses based on viewing the programs 
broadcast on television; ( 2 ) selected college, high school and adult 
learners were shown the films in small groups and were asked to complete 
a different reaction form at that time. 

The mai l questionnaire provided reactions to each of the five programs, 
while the film screenings with learners were limited to two programs 
(this was necessary because of the relatively time-consuming task of 
obtaining prints, shipping them to different locations, and using in- 
structional time to obtain feedback). Two films were selected for 
projection to represent opposite poles in terms of ease or difficulty 
of the subject matter. The overall mail responses indicate conclusively 
that, had it been possible to project and test student reaction to the 
other three programs, responses to them would have fallen somewhere 
between the responses obtained on the two programs selected. 

Phase I: Questionnaire for Broadcast Viewers 

In the first three months of 1969, mailing lists were obtained from 
two sources to mail quest ionnaires’to science teachers at secondary 
schools and at colleges and universities. 
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From the National Registry, National Science Teachers 
Association, NEA: 

10,000 biology teachers, high school faculty 

10,000 earth science teachers, high school faculty 

10,000 heads of science departments, high school faculty 
TOTAL: 30,000 high school teachers of science 

From the Educational Directory of Marion, Ohio: 

.3,230 biologists, college and university faculty 

2,682 geologists and meteorologists, college and university 
faculty 

802 astronomers, college and university faculty 

TOTAL: 6,71’+ college and university teachers of science 

GRAND TOTAL: 36,71* high school, college, and university teachers of 

science 



Each mailing included a questionnaire, one or two study guides which 
corresponded to the addressee's field of specialization (see Appendix B) 
and a postage-paid reply envelope, coded to the mailing list used. The 
mailing was timed to reash most recipients two weeks to ten days prior 
to the first scheduled broadcast. 




o 



Phase II: Evaluation for Learners in Test Groups 



(The College Sample of 150 students in 6 groups) 

Late in 1968 Peter C. Benedict, Associate Professor of Geology on the 
Albany campus of the State University of New York (SUEYA) was asked to 
read the study guides and to screen the films, in order to design in- 
struments of evaluation. Professor Benedict also agreed to conduct 
some of the evaluations with his students and arrange for others to be 
conducted by a colleague on the SUNYA faculty. 

(The Adult Sample of l60 adults in 8 groups) 

Shortly thereafter, arrangements were made with Milton Stem, Director 
of the University Center for Adult Education in downtown Detroit to con' 
duct similar tests with several groups of adults enrolled in various 
courses conducted by the Center. 

(The High School Sample of 195 students in l 6 groups) 
Also, agreement was obtained from Daniel Woodard, Vice Principal at 
White Plains High School, White Plains, New York, to conduct the tests 
with groups of high school students there. 

The group screening and testing phase began in February and ended in 

Miy, 1969. 
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RESULTS 



The portion of the report which follows falls logically into two main 
sections, one on the mail quest 5 ormair - and cue on the test groups, 
followed by some general conclusions. 

The findings and conclusions reported here may be supplemented and 
refined through subsequent research and replication* The raw data 
contained in nearly 1, 000 test papers and questionnaires obviously 
number in the tens of thousands and may be analyzed in many ways. 

It is our intention to make them available tc an educational researcher 
at the Indiana University Audio Visual Center who is working in the 
area of the evaluation of messages in educational materials* We hope 
that his work will develop important additional dimensions, as it is 
integrated with the current state of the art in ’ product evaluation* ” 
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Results: The 1*1*11 Questionnaire 



This phase of the evaluation was clearly the more innovative of the two 
procedures, for several reasons. First, it was geared entirely to television 
viewed at home, largely outside of school hours. Second, although addressed 
to professionals, it was designed to gauge the program* s relevance to 
laymen. Third, without offering an inducement other than an offer of 
additional study guides, it asked for a greater effort on the part of the 
respondent than most surveys: the respondents were required to ascertain 

the actual local air time for the program they were to see, they had to keep 
that time in mind for a period ranging from ten days to several weeks, make 
the effort and take the time to view one or more of the half-hour programs, 
and then communicate their reactions to the investigator. 

In order that this task not appear overly formidable, it was decided to ask 
each respondent to review just one, or at the most two, of the programs 
closest to their indicated field of specialization. Accordingly, most of 
the biologists received only the study guide for the genetics film, the 
geologists received the seismology guide and the meteorology guide or just 
one of the two, the astronomers were sent both the radio astronomy guide 
and the one on the Harold Urey interview, and the science department heads 
received the Harold Urey guide and any one of the other four* 

It should be noted in this connection that h2 $ of the respondents took the 
trouble to request one or more of the guides that had not been sent to them 
originally — clearly an indication of the usefulness of concomitant materials 



to aid study 



Overall Responses to Questionnaires 



As indicated above, a total of 36 , 71^ names and addresses were used in 
mailing the questionnaires. Normal attrition, through spoileage of 
labels and envelopes, brought this total down to 3^*986 pieces actually 
mailed. Direct mail specialists estimate that roughly 10$ of a mailing 
of this size is undeliverable due to address changes, deaths, less, 
and other causes. On that basis it can be assumed that between 31*000 
and 31*500 persons actually received the mailing. 

Since most of the mailing lists were national in scope (only one of the 
three NEA lists was limited to six "Major markets,") it must be 
assumed that a certain number of questionnaires were received in areas 
not yet served by public television. An estimated 20$ of television 
households in the nation are not served by public television stations, 
but that figure cannot be used because 65 responses were received from 
persons who returned the questionnaire with the comment that their area 
is not yet served by public television, many of them adding urgently 
worded remarks regretting their exclusion, or exhorting NET to extend 
its coverage. Despite this, it is fair to assume that an unspecified 
number of questionnaires, received in non-public television areas, were 
ignored for that reason. These must be added to other factors, such 
as the time and effort required for a meaningful response, and the local 
variations in air date and hour, in gauging the size of the response 
and its significance. It should be noted that the response to our mail 
questionnaire may have suffered in some locations because we were not 
able to inform all recipients of the exact day and hour they could 
view the program locally. 



Altogether, 420 responses to the questionnaire were received. This is 
between one and two percent, considered an acceptable rate of return 
from any sample as free from organizational ties to the agency originating 
the survey as were these teachers, and lacking any tangible benefit. 
Moreover, some of the respondents, on their own initiative, asked their 
classes to view the programs and reported a substantial number of reactions, 
rather than just one response. Some teachers went even further, asking 
their students to write reports on the program they had viewed, and sub- 
mitting these. Some of these student papers are in Appendix E. 

Some aspects of this investigation — among them the unsolicited student 
papers— must be viewed as a bonus. Other aspects which might have been 
anticipated but were not, made precise tabulation more difficult. Among 
these was the factor that many teachers returned the questionnaire, but 
did not use the coded reply envelopes, while others used the envelope to 
return a letter of their own but no questionnaire, and still others wrote 
letters without using either the questionnaire or the envelope. Finally, 
some useable responses were received from persons who could not see the 
program, but wished to comment on the study guides, or on some relevant 
aspect of public television. An attempt has been made to report all 
relevant information , despite the fact that these findings will not be 
easily categorized and that some totals, obviously, will add up to more, 
and some to less, than 100$. 




- 9 - 



Table 1: Overall Reaction to all 5 Programs: * 



Favorable 


llj-3 


57$ 


Mixed* ** 


12 


29$ 


Unfavorable 


35 





*See also the "Selected Comments" on each program. Appendix C 

**In most cases, respondents actually listed several specifics each 
under "strong points" and "weaknesses" 



Effectiveness of Mail Survey 



The most obvious statement that can be made here is that college and 
university instructors responded in far greater relative numbers than 
did high school teachers. Numbering only 5,6lQ, or 1 6 $ of our total 
mail count, they returned 127 coded replies out of 3k2, or 37$ of the 
response. This is further increased by the fact that the great majority 
of those using their own envelopes instead of ours were college or 
university instructors. 

There- may be significance in these figures beyond the scope of this 
report, apparently supported by other findings about the audiences 
of public television. Virtually all previous studies of that audience 
have agreed on one point: the more education one has, the more likely 

he is to be a regular viewer of public television. It is not un- 
reasonable to suppose, that, among those receiving our mailing, the 
ones most likely to respond were the more highly educated, and there- 
fore, regular viewers. 

The rate of response to all three of the high school teacher lists was 
below 1$ while it was between 2$ and 3$ for the college and university 
lists. The smallest of these lists, 7 88 astronomers, produced the 
highest return in the sample, but this may have been due to the fact 
that the radio astronomy film was the first program broadcast in the 
scries, and the only one not broadcast previously. On the other hand, 
the largest number of response was received for the genetics film 
(shown fourth) the second largest for radio astronomy (first in the 



series) followed by seismology and weather. The smallest response 
was that of the Harold Urey program, but this may be explained by the 
timing of the broadcast immediately prior to the Memorial Day 
holiday. 



Analysis of the Responses to Individual Questions on the Mail Questionnaire 



1* Date of Broadcast ; .This question was included as a check on the 
stations* scheduling of these programs* An analysis of the dates given 
shows that fully one-third of the respondents saw the programs at a time 
other than the network feed, and about half of these with a one-day delay, 
when the Eastern Educational Network of more than 25 stations carried 
them* A number of Western stations apparently aired "Spectrum" on 
Sunday, four days after the network release, and there was a sprinkling of 
still other dates as well* Some respondents were vague as to the exact 
date, and a small number appeared to respond on the basis of the original 
1967/68 airing of four of the programs. 

2. City in or near which broadcast was seen ; The geographic spread 
of the survey was comprehensive with responses from 44 states and the 
District of Columbia. (There is no public television coverage in three 
of the six states not responding.) 

3* Is it possible that you might have viewed this program if you had 
not received this notice? This question, designed to show whether the 
respondents were aware of public television, was answered as follows: 

yes: 60 $ no: 40$ 

It is worth noting here that, when an almost identical question was 
asked of the various test groups, the answers were: 





yes: 


no: 


High School Sample 


37* 


63 * 


College Sample 


32* 


68* 


Adult Sample 


77* 


23* 
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While this indicates a tremendous bias in favor of public television on 
the part of the adult sample, it is not inconceivable that persons who 
voluntarily take adult education courses, most of them non-credit, would 
also be the most likely group to view public television regularly, and be 
favorably disposed toward it. It is also consistent with what is known 
about the viewing habits of young people that students answered as they • 
did. Both student groups (high school and college) were unlikely to 
have viewed these programs as they were broadcast. High school students, 
generally living at home, had a slightly better chance to have seen them 
than did college students, a segment of the public which is known to see 
little or nc television of any kind. 

4. Do you feel that this program presented information about science 
that the general public should have? Overwhelmingly, this question was 
answered in the affirmative. The only negative responses came from those 
who felt that the particular subject matter was too complicated in its 
substance or presentation to be understood by the general public. Since 
the following question deals more specifically with comprehension, we 
had hoped by this question to learn whether teachers in general feel that 
scientific information should be made widely available, or whether it 
should remain the province of the expert. Clearly, they favor broad 
dissemination of scientific knowledge to the general public. 

5 . Was the information presented so as to benefit a lay person? In some 
respects, this was the central question for the entire survey, and the 
real answers to it must be sought throughout the written comments (see 
Appendix C) and in the letters. Expressed in figures, the answers were: 



Table 2: Presentation Suitable to Lay Public: 



Entirely : In Fart : 

Universe 15 3^ 

Weather IT 22 

Trembling Earth IT 20 

Genetics 15 T1 

Urey __T _8 

71 1 5T 



Not at All : 
1 
0 
0 
6 
0 
T 



6. Please comment briefly as to strong points and/or weaknesses in the 
presentation : Representative samples of replies are quoted in Appendix C. 

They speak for themselves, and dramatize the delicate balance, 'which the 
kind of program being evaluated must seek, between a body of knowledge 
that is vast, specific, and demanding and a method of presentation that 
must appeal to varied interests and motivations. The following table 
represents an attempt to summarize the reactions of all respondents to 
each of the programs inasmuch as the evaluation quotations in Appendix C 
do not reflect the actual distribution of positive, negative, and mixed 
responses. 



Table 3* Response to Individual Programs : 



Title- 


Favorable 


Mixed 


Unfavorable 


Total 


Universe 


29 


13 


juG 


52 


Weather 


23 


13 


k 


40 


Trembling Earth 


28 


8 


3 


39 


Genetics 


51 


35 


15 


101 


Urey 


12 




3 


18 




1^3 


72 


35 


250 



In additon to the comments in Appendix C Appendix E includes some letters 
and student papers of special interest, (included, as a tribute to the 
effectiveness of the mailing list, is a spirited letter from the renowned 
seismologist Charles F. Richter of the California Institute of Technology. 




- 15 - 



Analysis of Comments 



It is hoped that the comments (in Appendix C) convey the range of reaction 
received from the sample of professionals. Many seem to focus on whether 
or not the programs could he understood hy the lay viewer. A comfortable 
majority of individual statements, supported by the weight of the answer 
given to ques cion 5, ("Was the information presented so as to benefit a 
lay person?") seem to indicate a positive result. It appears that, except 
for more diagrams and animated sequences, the producers could not have 
done a great deal to make these subjects more accessible without compromising 
the substance that had to be transmitted. 

In addition, there is evidence that even the more difficult programs were 
reasonably well understood by 10th grade students and some 8th graders, 
viewing on television at home rather than in the classroom. In fact, it 
does not appear from this survey that those who viewed the films projected 
in the classroom were significantly better able to understand them than 
were the viewers of television. The kinds of detail, both on content and 
on production values, th£.t were observed by television viewers, indicate 
that the medium is fully capable of conveying the information required 
bv these subjects. 



y. In your opinion, could the broadcast have the effect of enc curbing 
young people to seek careers in science? Consistent with earlier expressions 

of approval, this question was answered as follows: 

yes: 188 no: 3^ 

Here it is appropriate to quote the answers given by so&e tenth graders 

in an honors section in BSCS(green version) Biology: 
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"Yes, the overall protrayal seemed honest." 

"Yes, unless they hadn’t planned on as much work." 

“Yes." 

"Possibly, if they had a sincere interest in science." 

"It could." 

"To seme degree." 

"Yes. Programs such as this tend to spark one’s curiosity. 1 

These comments were made about the program "Stop or Go," clearly the 
one found most difficult by all of the adults, teachers and laymen 
alike. (See section II, the Test Groups.) 

8 & 9* ( Optional space for name, address, and request for additional 
study guides .) As reported, h2.fo of all respondents did ask for additional 
study guides. Some, irdeed, went beyond the offer we had made and 
asked for large numbers of guides for one or several classes. Some 
of these requests were met at the outset, but it was found necessary 
later to enclose a note explaining that requests had to be limited to 
a single copy. 

As expected, most of the requests for guides came early in the five 
program series, dropping off toward the end. One kind of response 
occurred frequently in the beginning- -a request for additional guides 
and questionnaires in order to send in evaluations of subsequent 
programs. The investigator responded to these requests, using a special 
code on the return envelopes, but was disappointed to find that they 
were ro turned in only one case. Perhaps, in the final weeks of the 
school year, teachers found it impossible to carry out their earlier 
good intentions. 
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IX. Results: Learners In Test Groups 



The test groups included three samples: 

'Jhe College Sample of 150 students in 6 groups 
The High School Sample of 195 students in 1 6 groups 
The Adult Sample of l60 adults in 8 groups 

In general, the procedure was the same for all the test groups. The films 
were projected and the questionnaires handed out and completed during 
a regular session of the class or group being tested. Except where the 
instructor himself administered the test, advance permission was obtained 
from the regular instructor and the person conducting the test did so as 
a guest of the instructor. In Detroit, the adult students were given 
advance notice of the film showings and the test. All of the groups 
understood clearly that their participation was regarded as a courtesy to 
I®T and that their scores on the tests would have no significance whatever 
in terms of their standings in the regular course they were taking. 



Humber and Composition of Test Groups 

The College Sample (SUNY, Albany) 



Geology 105 
Geology 105 

Science 112 
Science 113 
Earth Science 202 
Earth Science 202 



21 students 

22 students 

31 students 
students 
l6 students 
l6 students 
150 college students 



Genetics 
Trembling Earth 

Trembling Earth 
Genetics 
Trembling Earth 
Genetics 



Students in Geology 105 are freshmen who may major in science; Science 112 
and 113 are service courses taught for non-science majors; Earth Science 
202 has an enrollment of juniors and seniors majoring in the sciences. 



The High School Sample (White Plains H.S., H.Y. ) 



10 Biology Lab Sections 

(Juniors and Seniors) ...... 105 Students . Genetics 

6 Earth Science Classes 

(Juniors and Seniors) 90 “ . . • . The Trembling Earth 



The Mult Sample (University Center for Mult Education, Detroit) 



fundamentals of Film Production 
(Credit Students) 

Fundamentals of Film Production 
(Credit Students) 0 

Contemporary Thought and 
Writing (Housewives) 

Contemporary Thought and 
Writing (Housewives) 

Basic Writing Class (ATTAC, 
Poverty Program) 

Beyond 60: Sounds of 

Silence (Senior Citizens) • • • • 

Writing and Language 
(Saturday Class) • • 

Graduate Students in 
Education (Univ. of Mich.) • • • 

total 

Grand Total 



15 


Students 


• .The Trembling Earth 


11 


It 


Genetics 


26 


11 


• .The Trembling Earth 


27 


11 




.15 


11 


• .The Trembling Earth 


20 


11 


• .The Trembling Earth 


32 


11 


. .The Trembling Earth 


14 


11 


• .The Trembling Earth 


1 S 0 


Adults 





505 Subjects in all samples 
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Analysis of Reaction Forms — College Sample and High School Sample 



All of the test groups operated under the necessary handicap of the 
paper-and-pencil instrument and *t»ie "testing situation" associated with 
it. The results obtained should not, however, he interpreted in the same 
way as test papers used in class instruction. The "passing grade" ob- 
tained in the usual students* tests signifies responses to a complex, 
purposeful process which a single television program (film) cannot be 
expected to replicate. Instead, when one considers the conditions under 
which most viewing of television takes place, it seems more appropriate 
to assume that, because of the technical nature of the information, 
knowledge was at a zero level before viewing the film. In that case, 
anything above chance response indicates knowledge gain which is logically 
significant, even where it is not statistically significant. This 
factor is underlined when one considers the probable numbers of persons 
viewing these films on television — likely in excess of one million for 
each film. 

With populations of that size, even the most random learning represents 
significant information transfer, and the evidence indicates that the 
learning which took place was far more than random. 

To me asure the learning effectiveness of the films, a simple scoring 
system was used, in which one point was awarded for giving a really 
relevant answer, one half point for showing some understanding or giving 
a partially correct answer, and zero for giving a wholly incorrect 
answer or no answer. * 

*For the "Trembling Earth" reaction form, see Appendix D; for the 
"Stop or Go" reaction form, see Appendix F. 
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With one exception, the best groups achieved "better scores on “The Trembling 
Earth” than on “Stop or Go," probably because the former was both easier to 
understand and provided more opportunities for emotional identification. 
Throughout all the groups, surprises were at a minimum, with higher scores 
being achieved by more advanced students (and, among the adult sample, by 
persons with more schooling). 



It is a matter for speculation whether scores could have been significantly 
improved by the inclusion of more diagrams ^desired by many respondents 
to the mail questionnaire, ) or by having the film provide a summary or 
"wrap-up" feature which students have come to expect in the more didactic 
films produced for classroom use. Whether or not this is the case, the 
requirements of television presentation had to govern the format used, 
calling for a non-didactic approach. 

In the context of television presentation, special attention needs to be 
paid to the "opinion" questions on the reaction form, and their possible 
relation to knowledge gain as seen by the viewer in the light of his 
original interest and the loss, maintenance, or increase thereof before 
and after viewing. 

Thus it is possible to compare the answers to Question 1, which asked if 
the respondents would have viewed the film on thoir own at home, and 
Question 3> which asked if the film had held the viewer’s interest. 

The following table traces this dimension for a sample of the high school 




students for each of the two films. 
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Table 4; Original Interest and After-viewing Interest 

( White Plains High School) 



Question 1 Question 3 Percentage of Sample 







Seismology 


Geneti< 


No 


Yes 


48$ 


36$ 


Yes 


Yes 


32$ 


22$ 


Yes 


No 


10$ 


2$ 


No 


No 


10$ 


38$ 



It is clear that the seismology film fared better with this sample than 
did the genetics film. Distortion and bias are evident in the number of 
students who said (after viewing,) that they would, or would not, have 
watched the film on their own. For the more 11 successful" film, 58$ said 
they would not; for the less successful, 7o$ said the same. They were pro- 
jecting their lack of approval into their estimate, although it might be 
argued as well that more young people would be inclined to view a tele- 
vision program on earthquakes than one on genes. 

However, the more encouraging evidence seems to be that whatever the 
estimate of original interest, majorities of both groups said that the films 
did hold their interest when they saw them: 80$ said so for seismology, 

60$ for genetics. The combination expressing the most disappointment — 
those with original interest who lost it when viewing the film— is so low 
in both cases that it is probably -without atistical significance. 

In a cross section of the college sample, where higher motivation may be 
inferred, the two dimensions of Yes/No and No/No did not show up at all 
for the seismology film, and Question 3 got an almost unanimous Yes 
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answer. 



By contrast, the genetics film ran into the same bias described 
for the nigh school sample — two thirds answering No to Question l, and 
the after- viewing approval was down from 100$ to 75$ * 



Achievement with respect to the content questions appears consistent with 
the foregoing observations. Reaction forms were graded and a Mean Score 
used throughout the samples. Correct and relevant answers to each 
question would result in a score of 100. 



Table 5: Mean Scores Achieved in Test Groups 

(High School and College Sample) 

S eismology 

High School Sample (ll:^7) 

High School Sample (N : 39 ) 

College Sample 

(Freshmen, General Science; N: 3l) ^2$ 

College Sample (Freshmen, Geology; N : 22 ) 5 W 

College Sample 

(Juniors and Seniors, Geology; N:l6) 



Genetics 



3 6$ 



7T$ 



In structors have reported that, except for the college juniors and seniors, 
none of the students had had instruction paralleling the content of 
either film. They were, in fact, thought to be representative of the 
average viewer of public television in educational attainment, if not in 
age. The college seniors scoring 77$ on the genetics film, even though 
their major was geology, represent a group of advanced science students 
who were easily able to handle a subject outside their major field, an 
indication of the perimeters of utility of this particular film. 
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The significance of these scores, as expressed toy one consultant, is that 
11 ••.if the viewers of these films are atole, on the average, to show an 
understanding of atoout half of the many points discussed, then the film 
must have "been effective. Also, if the film maintains the interest of 
people who would not have tuned their sets to it on their own, this shows 
what impact it could have cn the general public, if they could toe 
persuaded to view it." The same consultant points out that "... in the 
time of one half hour a significant increase in imowledge atoout a highly 
complex subject was effected." 



Analysis of Reaction Forms — Adult Sample 



Because of its greater relevance to this survey, the adult sample will he 
discussed in more detail than the other two samples* 



When the repoit on the Detroit adult test groups was submitted*, the 
covering letter had this to say about "Stop or Go" : 

"I did take the genetics film and show it to two groups after 
which point I decided to discontinue showing it. The response 
of the first two groups was very hostile and negative, and I 
felt I would be taking unfair advantage of the respondents if 
I continued showing the film. I do have 38 completed 
questionnaires which you may have." 



This confirmed once again that few people are indifferent about this film — 
they either like it a lot, or they hate it. The investigator felt that the 
Center was right not to persist in further screenings, and decided to 
summarize the extant questionnaires. These findings will be reported 
following the data on the completed tests with "The Trembling Earth." 
Meanwhile, it should be kept in mind that "Stop or Go" yielded twice as many 
mail questionnaires as any other program, and that college juniors and 
seniors scored very high in the test following their screening. 



* With minor alterations, this is the report prepared by Mr. Raymond 
Zelazny of the University Center for Adult Education. The changes made are 
designed to incorporate data on three groups omitted in the report: two at 

the University Center who viewed "Stop or Go," and one group of graduate 
students at the University of Michigan who viewed "The Trembling Earth." 




- 25 - 



NET requested the University Center for Mult Education to survey at least 
100 adult education students. The questionnaire prepared by NET was used 
by all the students in the survey. A cross section of adult education 
courses was selected to roughly represent students who typically participate 
in various UCAE activities. The sample was selected by type of participant — 
university credit students, poverty program trainees, senior citizens, 
housewives, workshop and general instruction students. These people are 
presumed to represent a cross section of the adult community based upon 
UCAE experience. Their enrollment in a formal program of instruction is 
assumed to suggest a favorable disposition toward education. 

The instructor of each group was asked for permission to conduct the 
project with his students. Those instructors who agreed were asked to 
announce the project to their class the session before it was to be con- 
ducted. The survey was conducted during the regularly scheduled class 
period. Attendance was reported as normal for all groups. The group was 
given the following information: (l) the purpose, sponsors and general conduct 
of the study; (2) two kinds of questions appearing on the questionnaire were 
explained as follows: a) questions 1 through 6 were to reflect expressions 

of viewers opinion; and b) questions 7 through 19 were questions about the 
film’s content. The half-hour film was then shown. Questionnaires were 
distributed, completed jsn ' returned. This procedure averaged about 15 
minutes. 

In the tabulation (see appendix D) opinion questions were separated from 
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content questions. Various comments about the film’s interest are charted 
separately and appear in Table I of appendix D. The content questions 
were scored on the basis of one point for a correct answer; \ point for 
a partiallv but correctly answered question, or an answer which was 
partially incorrect; and no points for a blank or an incorrect answer. 

Table I of appendix D contains responses to the opinion questions and answers 
to the content questions. Correct answers to content questions were 
determined from the film and from a study guide provided by National 
Educacional Television. A tabulation of the scores is contained in the 
su mmar y of each group— Tables II-VI of appendix D respectively. Tables 
II-VT reflect each group’s response and performance. 



- 27 - 




Analysis of Responses to Individual Questions on the Reaction Form 

(Adult Sample ) 



"The Trembling Earth” 



Opinion Questions 1-6 




Question 1. Would you have watched "The Trembling Earth" on TV at home, if 
you haa known about such a broadcast? 

90 - yes IT - no 

Eighty-four percent (84$) of the respondents indicated that they 
would watch the program. *) 

See Master Tabulation, appendix D 

Question 2. Do you feel that enough explanations of terms and processes 
were offered in the film? 

78 - yes 30 - no 

Seventy-two percent (72$) of the respondents felt that the film 
provided enough explanation of terms and processes. 

See Master Tabulation, appendix D 

Question 3. Did the film hold your interest? (Please explain briefly why 
it did, or did not.) 

99 - yes 15 - no 

Adult education students indicated a general interest in the 
earthquake film. The students indicated that their interest 
in the film was held primarily by the use of vivid illus- 
tration, a current events topic, and a well organized pre- 
sentation. 



See Table VII, appendix D 

Question 4. In your opinion, which of the groups listed below would benefit 
most from seeing this film? (Geologists, Scientists other than 
geologists. The general public, Persons who might choose 
geology as a career, People who live in an earthquake -prone area. 

169 - Non-science 49 - Science 

See Master Tabulation, appendix D 



* When the two groups viewing "Stop or Go" are included, the response 
drops to: yes: 77$ No: 23$ 
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Do you feel you gained some knowledge about: a) the behavior 

of the earth* s crust; b) the interior of the earth; c) the 
way scientific information is gathered; d) the way it is 
evaulated , e) the nature of waves. 

The film offered a general knowledge gain. The gain was most 
prominently reflected in "the behavior of the earth's crust v 
and "the way scientific information is gathered." 

See Master Tabulation, appendix D 

Did you know what a seismograph is before you saw this film? 

74 - yes 34 - no 

Sixty-eight and four tenths percent (68.4$) of the respondents 
indicated that they knew what a seismograph was before the film 
was shown. 

See Master Tabulation, appendix D 
Content Questions 7-19 

Question J. If you did not know, do you know now what it is? 

20 - yes l4 - no 

Of the 34 respondents who answered no to Question 6 20 answered 
yes to Question 7« This should reflect an information point 
gain in Question 8 and 9* 

(Possible 40 points.) 

The highest possible score for Questions 8-19 was 12. The respondents 
produced mean scores as follows: (To the nearest tenth) College students 

7*3; general course and workshop students housewives 3*5j lower socio- 
economic group 2 . 5 ; senior citizens 2.4. The overall group mean was 4.l6. 



Question 5« 



Question 6. 



Most adult education students (84$) would have watched the program at 
home on television if they had known about such a broadcast. The vivid 
illustrations, well organized presentation and current news and values 
which held the viewers* interest. Terms and processes were adequately 
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explained for most of the respondents (72$), "but these explanations were 
not translated into answers to content questions as reflected hy the total 
mean score (4.l6 out of a possible 12). The respondents indicated that the 
film would benefit the non -scientist more than the scientist (more than 3:l)« 
The film provided most information about the behavior of the earth T s crust 
(4: l) and the way scientific information is gathered (7^l)* Although many 
respondents (60$) indicated that they had a specific information gain 
(Question 7) they were unable to translate that gain into written answers to 
specific questions about the information (Questions 8-9). 

One of the problems with an evaluation such as this is that non-credit stu- 
dents are generally not accustomed +o taking tests whereas the credit student 
is. This suggests a pre- disposition toward the pencil-paper instrument 
provided by BET which would favor the group accustomed to tests. 



The scores from the content portion of this survey bear out the contention 
by showing a substantially higher average score (7 l/3) by the credit group. 
Also, xhe specific information gain question (7) indicates that the one 
individual who did not know what a seismograph was before the film knew 
what it was after. This was demonstrated by the information gain point 
increase of two out of a possible two. 

The opinion expressed in Question 2 was that an adequate explanation of the 
terms and processes was offered. This is contradicted by the respondents* 
inability to record answers about the film’s content as evidenced by the 
mean score of 4. ±6 out of a possible 12. Based upon the instructor s 
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experience :qtv ^- assessment of the film, it was predicted that the final 
rank order of the groups would he credit students, housewives, general 
audience, senior citizens and poverty program people. The post-survey 
outcome ra nk ing conformed closely: credit students, geneial audience, 

housewives, poverty program people, senior citizens.* A radical change 
in this r ank ing or a closer grouping of scores might indicate a greater 
effect on learning due to the film. 

Table 6: Mean Scores Achieved in Test Groups 

(Adult Sample] ~~ 



Credit Students (Film Production; 11:15) 

Graduate Students (Education, U-jfM; N:l4)* 

Writing and Language (W:32) 

Credit Students (Film Production; N:ll) 

Housewives (N:26) 

Poverty Program (N:15) 

Senior Citizens (N:20) 

As reported oy Mr. Zelazny, these groups somehow could not come to grips 
with the generic s film. The group for which a score was obtained (Credit 
Students,) actually did far better even with this film than some of the 
educationally disadvantaged groups did with the easier film, although its 
own score dropped by 24$. Yet, when compared to reaction forms the same 
individuals had completed f?/r the seismology film, these forms expressed 
frustration, even irritation. This was even more marked for the only other 
group that was shown tne genetics film, the housewives, xn both groups-, 
the handwriting appeared more careless, and comments such as too scientific 
and technical," "too deep," abounded. 

* Hie graduate students* group was not part of Mr. Zelazny* s assignment. 
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Seismology 



Genetics 



60$ 

59$ 

1 , 2 $ 

29$ 

21 $ 

20 $ 



36 $ 



o 



The two groups viewing both films also showed marked variations in the 
opinion questions. Their combined answers to Question 1, (prior interest,) 
were: Seismology — Yes: 32, No: 8. Genetics— Yes: 20, No: 18. For 
Question 3, (interest maintained,) the answers were: Seismology — Yes: 38, 

No: 3* Genetics: Yes: 1 6, No: 20. This represents the single instance 

in the survey where a majority reported that a film had not held their 
interest. 

In general, the scores obtained in the adult sample are far from dis- 
appointing. Some were actually higher than those of the student samples, 
while those that were lower came from groups of persons whose education 
was deficient. In addition, the adults as a group are, of course, far 
less accustomed to the testing situation than are the students. 
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CONCLUSIONS AND DISCUSSION 



It is evident that the five science programs broadcast by NET reached 
a high level of acceptability and information transfer. 

Both learners and professionals indicated that their attitudes toward 
the content and treatment of the various science topics were essentially 
positive, and were enhanced or maintained throughout the presen cation. 



One problem with this kind of evaluation is the relative lack of precision 
concerning ‘’instructional" goals. When educational films are evaluated 
for curricular integration, instructional utility is measured against 
precise spec if ications , keyed to specific objectives and courses of in- 
struction. In public television programming the problem is quite different. 
Not only is there no particular teaching objective or audience established, 
but the motivation of viewers may vary from a highly specific, active, pro- 
fessional interest in the subject to an "entertain me, if you can" passivity. 

In some respects the five programs, along with most others in the "Spectrum" 
series, tried to do two things considered by some educators to be opposite: 
we want to teach and we want to "entertain," both in an environment we do 
not control, and in which it is up to the individual viewer whether he 
chooses to be taught, to be entertained, or neither* This presents the 
television producer with a challenge and an opportunity. The challenge is 
to live up to reasonable professional standards; the opportunity is to appeal 
to large numbers of persons with the hope of awakening an interest in science. 

Even so, the question remains whether the film should, for example, increase 
by 10 percent the knowledge of a viewer already familiar w?.th the subject 
matter, or by 50 percent the knowledge of one never before exposed to it. 

This research seems to indicate that we were able to appeal to both kinds 
of viewers, well informed and uninitiated, but that these results varied 
strongly from one film to another. 

Our approach to this evaluation has been eclectic; there are few precedents 
to guide us. Even in the seemingly more precise context of purely 
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instructional films and television programs there are few absolutes, as 
evidenced in this research rationale for the Educational Products 
Information Exchange by Robert E. Stake*: 



Ho product evaluation can be complete without a survey of the 
preferences and priorities of the many groups who use the product, 
or who may benefit or be injured by it. 

• • • 

Every product can, of course, be described in a variety of ways, 
and comparisons among products can be made on many different 
grounds. Two dictionaries, for example, may differ as to 
number of words defined, size or type, durability of binding, 
and attractiveness of illustrations. They may differ, too, in 
less tangible matters, such as the thoroughness of definitions 
or the sanctity in which formal grammar is held. One dictionary 
is likely to be better for some purposes, another for other 
purposes. It will be the responsibility of the researcher to 
describe the dictionaries as fully as he can, then to indicate 
the conditions under which he knows or suspects that individual 
dictionaries will do a good job (and, sometimes, which dictionary 
will do a better job.) 



We felt justified in going our own way to try to discover something about 
the "instructional utility" of these programs, and also to seek information 
about viewing habits and attitudes toward science on public television. 



The role of public television in disseminating scientific information to 
the many audiences it serves is affirmed in the results analyzed in this 
investigation. Films such as these in the "Spectrum" series are documented 
reports of current research in basic and applied science. They utilize 
the techniques and expertise of, on the one hand, the scientists them- 
selves, and on the other, the professionals of public television. The 

*Excerpts from A Research Rationale for EPIE, by Robert E. Stake, 
Educational Products information Exchange; The EPIE Forum. 

Vol.l, No.l, Sept., 1968, pp. 7 & 8. 
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"mix" produced by this collaboration amounts to an unexcelled, highly 
communicative learning opportunity lor millions of persons of greatly 
varying backgrounds and levels of sophistication. The word "opportunity” 
is central, for public television does not "teach" in the conventional 
sense- Rather- it provides opportunities for learning, blueprints for 
further involvement. Many viewers seek these consciously, but many more 
receive them subliminally. In some the effect may be sharp awareness, 
even genuine knowledge. In others it is more tentative, a sampling to 
which they may return later. 

As viewing of public television increases, it is inevitable that the 
intellectual and aesthetic tone of the nation’s communities is raised 
and the public helped in making wiser choices in crucial issues. Clearly, 
science embodies many such issues, and filmed reports such as these are 
fundamental to the existence of an informed and enlightened public. They 
thus perform a service of high priority, and one which may not be similarly 
available in any other medium of communication. 



Appendix A 



Mail Questionnaire 



I 




mi : o to: 



From: 



NATIONAL EDUCATIONAL TELEVISION 

10 COUJMDUS CIRCLE NEW YORK. NEW YORK 10019 (212* 202-4200 

Science Faculty and Administrators 
Educational Servicer , I JET 



April 1969 



Subject: Evaluation of five science programs 

Searching for a mysterious source of er ergy in space; trying to turn 
a harmless cloud into a rain storm; discovering how to predict when disastrous 
earthquakes will occur; reading the “language" of genetics, and visiting a world- 
renowned scientist — fnese are experiences in store for viewers of five half-hour 
programs to be broadcast this spring over most of the NET network. 

Produced irith financial assistance from the National Science Foundation, 
the programs are, in the order of broadcast: 



Exploring the Universe — in Piacio and Light 
Changing the Weather (Color) 

The Trembling Earth (Color) 

Stop or Go — an Experiment in Genetics (Color) 
A Visit with Harold Urey 



.^ e( 
^ A 



The network broadcast schedule for the five programs will be weekly, 
beginning at 8:0C p.n. Eastern Time on April 30, and ending on May 28. However, 
local ETV listings should be consulted in every esse, as stations may delay (or 
repeat) programs in many areas of the country. 

NET asks for your help in evaluating the effectiveness of these programs. 

We are especially interested in learning whether they can help bridge gaps in the 
lay public's understanding of science. Please help us by returning this form after 
viewing at least one of the five broadcasts. A return envelope, requiring no postage,] 
is enclosed for your convenience. ' | 

You will also find enclosed one or more study guides of programs in 
your indicated field of specialisation. Please use them as a means cf acquainting 
yourself and your students with the program. In your evaluation, however, please 
bear in mind that the average home viewer will net usually have such a guide available. 






NET attempts, in its continuing science programming, to present infor 
mation in such a way that the lay public's appreciation of science is increased. 
Although we feel that these television programs can also enrich the science 
curriculum, the principal aim is to provide information to the viewer at home. 



You can help significantly in this effort by taking a few minutes to 
give us your opinion of the approach we have taken, and any suggestions as to hew 
it might be improved. 



If you are able to view more than one of the five programs, and woulc. 
be willing to give us your reaction to them, pleasa fee], free to use additional 
sheets. The use of your name and/or address is optional. 

May we thank you in advance for your cooperation. 





(please turn page) 



w * 



i i * 



C. I* d i-icnt! 'aril forv *c IIEP 



Subject: U/uini ion ,t‘ 



'i'ltio vi* Program 



1. bate of broadcast: 



2. City In or near rhieh h-cvle&rt vas seen: 



3 . I:; it possible ~\vjlz you might have viewed this program if you 



had not received this notice? 

yes 



no 



It. Do you feel that this program presented information about 
science that the genera?, public should have? 






. Vise the inf or nation orcienfed so as to benefit a lay person? 



Entirely 



in Part 



Not at AIL 



o . 



Please cctanent briefly ss to strong points and/ cr weaknesses 



in she •oresencaxion: 



T- 



In your opinion, 
yoar.g people to 



could the 
seek career 



broadcast have the effect 
s in science? 



of encouraging 



a 



(OPTIONAL) Your name: 



Address: 



Q 



If yru like single copies of oiu.ly guides not enclosed, please 

’'job Lov .■ sire titles you uar.v (.and compa-cte your address . ) 




